1. Background Field
Embodiments of the subject matter described herein are related generally to image processing, and more particularly compensating for the lack of a time stamp when images are captured.
2. Relevant Background
Computer vision based tracking methods suffer from a lack of robustness when there are quick motions and, in particular, when there are large image displacements caused by the rotation of the camera. Hence, aiding computer vision tracking with inertial sensors can drastically improve the tracking robustness. Sensor fusion between poses derived from camera frames and inertial sensor measurements can only occur if accurate time stamps for camera frames and inertial sensor measurements are available.
In current mobile device platforms, the application program interface (API) for inertial sensors includes a time stamp field and the inertial sensor data is time stamped when the sensor is sampled. Unfortunately, many current mobile devices do not have a time stamp field in the camera API. Accordingly, in such mobile devices, image frames are not time stamped at the sensor level when captured by the camera, but are time stamped at a later time, e.g., when delivered to an application level, such as at a High Level Operating System (HLOS) or other level. In addition, sensor measurements are often low pass filtered to suppress noise. Sometimes, the low pass filtering is aggressive, effectively averaging measurements over more than one sample and thereby rendering the sensor time stamp inaccurate. Thus, fusion between poses derived from camera frames and inertial sensor measurements is inaccurate.